Skip to content

docs(skills): improve drafting skills from signal log patterns 2026-08-01 - #450

Closed
oz-by-warp[bot] wants to merge 1 commit into
mainfrom
docs/improve-drafting-skills-2026-08-01
Closed

docs(skills): improve drafting skills from signal log patterns 2026-08-01#450
oz-by-warp[bot] wants to merge 1 commit into
mainfrom
docs/improve-drafting-skills-2026-08-01

Conversation

@oz-by-warp

@oz-by-warp oz-by-warp Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Patterns addressed

  1. content_structure (human feedback: review comments/verdicts across 5 PRs, plus prior log entries on error-message placement)
    • Reviewers repeatedly asked for reader-chronology section order (requirements → setup → usage) and for error strings to live in a dedicated Troubleshooting section instead of the main flow.
  2. callout (human feedback: review comments + human edits across 6 PRs)
    • Agents over-used :::note / tip callouts; humans trimmed stacked callouts in favor of body prose.
  3. settings_path orientation (human feedback: human edits across 7 PRs)
    • Path bolding already existed in skills; remaining gap was naming the app/tool before the first Settings path, CLI command, or URL on the page.
  4. image_alt / screenshot discipline (human feedback across 4 PRs)
    • Reviewers pushed back on unnecessary screenshots and internal-only UI captures; alt-text checklist existed but placement/when-to-use guidance was thin.

Signal window: last 30 days. Primary source: GitHub human review comments, review verdicts, and post-agent human edits on 59 agent-coauthored merged PRs. Oz [SIGNAL:style-lint] / [SIGNAL:pr-review] markers: 0 found in 36 drafting-related run conversations (inner loop is not yet emitting markers reliably).

Improvement targets

  • .agents/skills/draft_docs/SKILL.md — additive Critical formatting rules + checklist items for section order, Troubleshooting placement, callout sparsity, Settings/CLI/URL orientation, and screenshot discipline (applies to all drafting skills that route through draft_docs).
  • .agents/templates/feature-doc.md — bracket instructions for chronology, no errors in conceptual sections, optional Troubleshooting before Related pages, sparser callouts, stronger Related pages guidance.
  • .agents/templates/procedural.md — prerequisites-before-steps, app orientation, and Troubleshooting as the home for exact error strings.

Patterns reviewed but not acted on

  • terminology (14 PRs) — already covered by glossary + product name variables rules in step 6.5; remaining issues were page-specific accuracy, not missing skill text.
  • list_format (11 PRs) — already has explicit bold+dash rule and checklist item.
  • scannability (10 PRs) — already covered by tables/parallel-bullets rule and checklist scannability item from prior loop.
  • heading_specificity (4 PRs) — already has descriptive-headings rule with ✅/❌ examples.
  • link_quality (2 PRs) — Related pages already in templates; count at threshold but mostly one-off “add related links” nits.
  • general (20 PRs) — heterogeneous product-accuracy feedback; no single skill edit would prevent it.

Open questions for human review

  1. Is “at most one or two callouts per page” the right default, or should feature docs allow a third for enterprise/security caveats?
  2. Should draft_feature_doc get a type-specific Troubleshooting requirement (always include the section) rather than optional-but-recommended in the shared template?
  3. Inner-loop SIGNAL emission appears broken or unused (0 markers in 30 days). Worth a follow-up so style_lint/pr-review counts can feed this loop automatically?

Test plan

  • git diff --check clean
  • YAML frontmatter parse check on changed skill/template files
  • Human review of whether the new rules match intended style guide emphasis

Standing signal-log PR (separate): #433

Conversation: https://app.warp.dev/conversation/6def4ca6-a9a1-4158-8027-40a525ed499d
Run: https://oz.warp.dev/runs/019fbe44-b2bf-7599-972c-0e0aef3b8974
This PR was generated with Oz.

…8-01

Add additive guidance for section order, troubleshooting placement,
callout sparsity, Settings-path orientation, and screenshot discipline
based on human review patterns from agent-authored docs PRs.

Co-Authored-By: Oz <oz-agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label Aug 1, 2026
@vercel

vercel Bot commented Aug 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 1, 2026 5:17pm

Request Review

@rachaelrenk

Copy link
Copy Markdown
Contributor

Superseded by #487, which consolidates this PR together with the other stacked improve-drafting-skills PRs (#450, #454, #468, #484).

All four edited .agents/skills/draft_docs/SKILL.md and conflicted with each other, so none could merge cleanly. The patterns from this PR are carried over in #487 — overlapping rules were merged rather than stacked, and anything already superseded on main was dropped.

Root cause: the agent's schedule used 0 17 1-7 * 1, which is not "first Monday." Cron ORs day-of-month with day-of-week, so it fired roughly 11 times a month. #487 fixes the cron, adds a first-week guard, and gives the agent a single standing PR so this cannot recur.

@rachaelrenk rachaelrenk closed this Aug 6, 2026
rachaelrenk added a commit that referenced this pull request Aug 7, 2026
…ements (#487)

* docs(skills): reduce automation noise and consolidate drafting improvements

Recurring docs agents were producing more PRs and Slack messages than the
team could absorb. Three systemic causes, plus a batch of GitBook-era
migration artifacts that left several skills unable to run as written.

Shared conventions (skill-authoring-guidelines.md):
- Add "One standing PR per automation": stable branch and title, look
  before creating, add to the existing PR rather than opening another.
- Invert "Slack notifications" to actionable-only. The old rule required
  posting on every run and was the direct cause of the channel noise. Its
  silent-failure rationale is preserved by requiring a run log instead.
- Rewrite "Log availability" so outer loops read the log branch rather
  than main, and never merge the standing log PR as a workflow step.

Cron correctness:
- `0 17 1-7 * 1` is not "first Monday". Cron ORs day-of-month with
  day-of-week, so it fired ~11 times a month and produced four
  conflicting PRs in six days. Replace with `0 17 * * 1` plus an
  in-skill first-week guard in improve-drafting-skills,
  improve-aeo-crosslink-skill, and improve-404-monitor-skill.

PR reuse applied to: improve-drafting-skills, weekly-404-monitor,
afdocs-fix, sync-error-docs, sync_terminology, sync-openapi-spec,
improve-aeo-crosslink-skill, improve-404-monitor-skill. update-changelog
keeps one PR per release (correct) but now detects stacked release PRs.

Slack volume: aeo_crosslink_audit no longer posts on no-change runs;
weekly-404-monitor gates on threshold and folds its Phase 2 results into
a single message instead of two; afdocs-audit posts only on regression or
a blocked audit, backed by a new run log for the baseline.

Migration artifacts: a find-and-replace during the GitBook-to-Astro move
substituted descriptions into file paths. sync-error-docs referenced
`astro.config.mjs (sidebar config)` and `vercel.json (redirects)` as real
paths, had an invalid grep, and still called the GitBook API - it could
not have succeeded. Also corrected the sidebar location to src/sidebar.ts,
dropped the GITBOOK_TOKEN dependency, and fixed dead
`.warp/references/terminology.md` paths in four skills.

Consolidates PRs #450, #454, #468, and #484, which all edited
draft_docs/SKILL.md and conflicted with each other. Overlapping patterns
were merged rather than stacked, and PR #468's frontmatter-description
edits were dropped as already superseded on main.

Co-Authored-By: Warp Agent <agent@warp.dev>

* docs(skills): document the deployed monthly cron for improve-drafting-skills

The schedule was deployed as `0 15 1 * *` (the 1st of each month) rather
than the `0 17 * * 1` + first-week-guard combination the skill documented.
Both are correct and both fire exactly once a month, but the docs and the
deployed schedule disagreed.

Documented the deployed expression. Restricting only day-of-month is
unambiguous because day-of-week stays `*`, so there is no ORing hazard. The
tradeoff is noted: the 1st can land on a weekend, delaying review.

Kept the first-week guard as a safety net and explained why, since it no
longer trips on its own: it is what would narrow a day-of-week expression
back to the first Monday, and it contains the blast radius if the
day-of-month/day-of-week ORing mistake is ever reintroduced. Reworded the
guard's skip message, which still referenced 'first Monday'.

Co-Authored-By: Warp Agent <agent@warp.dev>

* docs(skills): treat a log-branch fetch failure as blocked, not a stale fallback

Review catch on improve-aeo-crosslink-skill: its step 0 said to fall back to
the log copy in the current checkout when the branch fetch fails. That copy
comes from `main` — precisely the truncated history the branch read exists to
avoid — so the fallback reintroduced the problem this PR set out to fix. The
file also contradicted itself: its Slack section already listed 'could not
fetch the log branch' as a blocked-run example while step 0 said not to abort.

The failure mode is quiet, which is what makes it worth fixing. A short log
still parses; only the counts change. The run then either drops below the
8-entry minimum and reports 'too early to analyze', or clears the minimum on
stale entries and proposes skill edits from an incomplete picture. Both look
like ordinary outcomes, so nobody investigates.

Both outer loops that read a log branch now stop before analysis on a fetch
failure and post the blocked-run message. improve-drafting-skills already had
this behavior documented and is unchanged.

Also generalized the rule in the authoring guidelines, since this is a class of
bug rather than a one-off: do not adopt a fallback that is quieter but less
correct than failing. The test is whether the fallback can change the answer
ratherratherratherratherratherratherratherratherratherratherratherratverage
transparently — proceeding on one source signal and recording the gap — remain
fine, because the reader can see what was missing.

Co-Authored-By: Warp Agent <agent@warp.dev>

---------

Co-authored-by: Warp Agent <agent@warp.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants